At present, the video key information extraction technology mainly focuses on the extraction of key frames according to the characteristics of video low-level, and ignores the semantic information related to users' interests. Semantic modeling of video requires a large number of marked video training samples, which is time consuming and laborious. To alleviate this problem, a large amount of Internet image data was used to construct a semantic model based on users' interests, which was rich in content and covered a large amount of event information. However, the images obtained from the Internet were diversed and often accompanied by image noise, the final extraction of video would be greatly affected by brute force migration. The synonym-weight model was used to measure the differences of the semantically similar image groups on the Internet, and these image groups were used to construct a semantic model. The weight of each image group in knowledge migration was determined by the weight value. The experimental results on several challenging video datasets demonstrate that semantic modeling based on users' interests combined with weights is more comprehensive and accurate, so as to effectively guide the video key frame extraction.